Mobile robot for sensing and decoding a surface coding pattern on a surface

ABSTRACT

A mobile robot for sensing and decoding a surface coding pattern on a surface includes a housing, A platform is in the housing. An on-board controller is mounted on the platform and configured for wireless communication with a host computer system. A differential drive system is mounted on the platform and connected to the controller so that the controller can steer the platform relative to the surface. An image sensor is mounted on the platform, arranged to sense the surface coding pattern and connected to the controller to communicate data representing the surface coding pattern to the host computer via the controller.

Cross Reference To Related Application

The present application is a Continuation of U.S. application Ser. No.11/754,370, filed on May 29, 2007, now issued U.S. Pat. No. 7,358,697,which is a Continuation of U.S. application Ser. No. 11/474,271, filedon Jun. 26, 2006, now issued U.S. Pat. No. 7,245,760, which is aContinuation of U.S. application Ser. No. 10/782,895, filed Feb. 23,2004, now issued U.S. Pat. No. 7,148,644, all of which is hereinincorporated by reference.

TECHNICAL FIELD

The present invention relates to a robot, and in one particular form, toa mobile or moveable robot that can behave in a particular way inresponse to the particular surface upon which the robot is placed andthe robot's position on that surface, and in another particular form, toa mobile or moveable robot adapted to mark a particular surface.

CO-PENDING APPLICATIONS

Various methods, systems and apparatus relating to the present inventionare disclosed in the following co-pending applications filed by theapplicant or assignee of the present invention simultaneously with thepresent application: NPS045US.

The disclosure of this co-pending application is incorporated herein bycross-reference. The application is temporarily identified by its docketnumber. This will be replaced by the corresponding USSN when available.

CROSS-REFERENCES

Various methods, systems and apparatus relating to the present inventionare disclosed in the following co-pending applications filed by theapplicant or assignee of the present application:

7156289 7178718 7225979 09/575197 7079712 6825945 09/575165 68130397190474 6987506 6824044 6980318 6816274 7102772 09/575186 66810456678499 6679420 6963845 6976220 6728000 7110126 7173722 6976035 68135586766942 6965454 6995859 7088459 6720985 7286113 6922779 6978019 68478837131058 7295839 09/607843 09/693690 6959298 6973450 7150404 69658827233924 09/575181 09/722174 7175079 7162259 6718061 10/291523 10/2914717012710 6825956 10/291481 7222098 10/291825 7263508 7031010 69728646862105 7009738 6989911 6982807 10/291576 6829387 6714678 66445456609653 6651879 10/291555 7293240 10/291592 10/291542 7044363 70043906867880 7034953 6987581 7216224 10/291821 7162269 7162222 72902107293233 7293234 6850931 6865570 6847961 10/685523 10/685583 716244210/685584 7159784 7068382 7007851 6957921 6457883 10/743671 70949107091344 7122685 7038066 7099019 7062651 6789194 6789191 6644642 65026146622999 6669385 6827116 6549935 6987573 6727996 6591884 6439706 67601197295332 7064851 6826547 6290349 6428155 6785016 6831682 6741871 69278716980306 6965439 6840606 7036918 6977746 6970264 7068389 7093991 71904916982798 6870966 6822639 6474888 6627870 6724374 6788982 7263270 67882936946672 6737591 7091960 09/693514 6792165 7105753 6795593 69807046768821 7132612 7041916 6797895 7015901 10/778056 10/778058 10/77806010/778059 10/778063 10/778062 10/778061 10/778057 7055739 72333206830196 6832717 7182247 7082562 6843420 10/291718 6789731 70576086766944 6766945 7289103 10/291559 7299969 10/409864 7108192 711179110/683151 10/683040 10/778090 6957768 09/575172 7170499 7106888 71232396982701 6982703 7227527 6786397 6947027 6975299 7139431 7048178 71180256839053 7015900 7010147 7133557 6914593 10/291546 6454482 68083306527365 6474773 6550997 7093923 6957923 7131724

The disclosures of all of these co-pending applications are incorporatedherein by reference.

BACKGROUND ART

Principles of mobile robotics are described in detail in Dudek, G., andM. Jenkin, Computational Principles of Mobile Robotics (CambridgeUniversity Press, 2000) and Nehmzow, U., Mobile Robotics: A PracticalIntroduction (Springer Verlag, 2000). Practical mobile robotconstruction is described in detail in Wise, E., Applied Robotics(Prompt Publications, 1999) and McComb, G., The Robot Builder's Bonanza,2nd Edition (McGraw Hill, 2001).

The present invention is not limited to use requiring a netpage.However, a netpage is now described to assist the reader inunderstanding the nature of the invention. A netpage consists of visiblegraphic data intended for human interpretation, as well as substantiallyinvisible (or at least inconspicuous) netpage coded data intended formachine interpretation. The netpage coded data identifies, at each of atleast a given density of points on the netpage, the identity of thenetpage and the two-dimensional coordinates of the point. The givendensity is typically of the order of a few millimeters. A netpagesensing device incorporates an optical sensor and a decoder for netpagecoded data. When placed in a position to sense netpage coded data, thenetpage sensing device is able to determine the identity of the netpageand its own position relative to the netpage from a purely localsnapshot of the netpage. Whereas the netpage coded data only directlyencodes position to a certain precision (e.g. of the order of a fewmillimeters), the netpage sensing device can determine its position andorientation relative to the netpage to greater precision (e.g. of theorder of a few micrometers) based on the alignment, rotation andperspective distortion of the coded data in its field of view.

Note that the distinction in a particular coding pattern between pageidentity (i.e. netpage id) and point coordinates is merely a functionaldistinction. An actual coding pattern may utilize any mixture (orhierarchy) of region identifiers and coordinates, ranging from a purecoordinate code where the coordinate space spans a multitude of pages,through to a pure identifier code where a page contains a multitude ofidentified regions. In the preferred coding, as described above, thecoding pattern contains a multitude of identified regions, eachcontaining a range of coordinates, and each conveniently correspondingto the size of a page. Note also that the size of a page is itself notfixed, and may correspond to the size of a sheet of paper (e.g.Letter/A4, Tabloid/A3, etc.), or to the size of the surface of anarbitrary object.

If the scale of the netpage coding pattern is increased (e.g. so thatthe given point density is of the order of centimeters or decimeters orlarger), then the required imaging field of view grows accordingly.However, the precision with which the corresponding netpage sensingdevice can determine its precision and orientation relative to thenetpage is a function of the device's imaging resolution, not the sizeof its field of view. It is therefore possible, given sufficientresolution, to determine position and orientation to arbitraryprecision, independent of the scale of the netpage coding pattern,subject of course to normal optical imaging constraints such asdiffraction limit.

A netpage may be printed onto a surface, such as the surface of a sheetof paper, using a netpage printer. Printing may take place in bulk, oron demand. The graphic data is typically printed using visible inks,such as cyan, magenta, yellow and black inks. The coded data istypically printed using an invisible ink such as an infrared-absorptiveink, or using a low-visibility ink.

More generally, the graphic data may be printed or otherwise depositedon or in the surface by any suitable device or process, and the codeddata may be printed or otherwise deposited on or in the surface by anysuitable process. The two devices and/or processes may be entirelyunrelated, and need not operate simultaneously. It is within the scopeof the present invention that the pattern of the coded data isrepresented in any way that allows it to be sensed, e.g. optically,magnetically, chemically, etc.

A netpage disposed on a surface is backed by a description of thatnetpage stored in a computer system, indexed by the netpage id. When anetpage sensing device interacts with a netpage, the sensing deviceforwards the details of the interaction to the computer system forinterpretation with reference to the stored netpage description. Theforwarded details typically include the decoded netpage id and thedecoded position of the sensing device relative to the netpage. Thedetails may also consist of a stream of netpage ids and positions. Whenthe netpage sensing device is in the form of a writing implement orstylus, and the stream is therefore representative of motion of thewriting implement or stylus relative to the netpage, the stream isreferred to as digital ink. The netpage sensing device then typicallyincorporates a contact sensor, and is configured to begin generating thestream when it comes into contact with the surface, and cease generatingthe stream when contact with the surface is broken. Each digital inkstream delimited by a contact event and a loss of contact event isreferred to as a stroke. The computer system retrieves the netpagedescription corresponding to the netpage id embedded in the stroke, andinterprets the stroke with respect to the semantics of the netpagedescription. For example, if the netpage description describes a textfield with a specific position and extent, the computer system maydetermine whether the stroke intersects the text field, and if so mayinterpret the stroke, in conjunction with other strokes similarlyassigned to the text field, as handwritten text, and attempt to convertthe strokes to a string of identified characters. If the netpagedescription describes an action zone (also referred to as a hyperlink)with a specific position and extent, the computer system may determinewhether the stroke intersects the zone, and if so may interpret thestroke as invoking the action, which may in turn cause the computersystem to send a corresponding message to another application associatedwith the action. Alternatively, a netpage stroke is forwarded directlyto an application which is specific to the netpage id embedded in thestroke.

If the netpage sensing device incorporates a marking nib or printingdevice, then the computer system typically associates digital ink inputfrom the device with the corresponding page by storing the digital inkin a persistent manner, indexed by the netpage id. In this way thedigital ink can be reproduced when the page is re-printed, and can besearched.

A netpage sensing device in the form of a stylus and pen is described inco-pending PCT application WO 00/72230 entitled “Sensing Device, filed24 May 2000; and co-pending U.S. application U.S. Ser. No. 09/721,893entitled “Sensing Device”, filed 25 Nov. 2000. A netpage sensing devicein the form of a viewer is described in co-pending PCT application WO01/41046 entitled “Viewer with Code Sensor”, filed 27 Nov. 2000.

Various netpage coding schemes and patterns are described in co-pendingPCT application WO 00/72249 entitled “Identity-Coded Surface withReference Points”, filed 24 May 2000; co-pending PCT application WO02/84473 entitled “Cyclic Position Codes”, filed 11 Oct. 2001;co-pending U.S. application U.S. Ser. No. 10/309,358 entitled“Rotationally Symmetric Tags”, (docket number NPT020US) filed 4 Dec.2002; and Australian Provisional Application 2002952259 entitled“Methods and Apparatus (NPT019)”, filed 25 Oct. 2002.

The reference to any prior art in this specification is not, and shouldnot be taken as, an acknowledgment or any form of suggestion that suchprior art forms part of the common general knowledge.

DISCLOSURE OF INVENTION

In one aspect the present invention provides a robot adapted to operatein association with an interface surface having disposed therein orthereon coded data indicative of an identity of the interface surfaceand of a plurality of reference points of the interface surface, therobot comprising:

-   -   (1) movement means to allow the robot to move over the interface        surface;    -   (2) a sensing device which senses at least some of the coded        data and generates indicating data indicative of the identity of        the interface surface and of a position of the robot on the        interface surface;    -   (3) communication means to:        -   (A) transmit the indicating data to a computer system, the            computer system programmed to select and execute a computer            application based on at least the identity of the interface            surface; and,        -   (B) receive movement instructions from the selected computer            application, whereby, the behaviour of the robot is at least            in part controlled by the selected computer application.

In another aspect the present invention provides a robot adapted tooperate in association with an interface surface having disposed thereinor thereon coded data indicative of an identity of the interface surfaceand of a plurality of reference points of the interface surface, therobot comprising:

-   -   (1) movement means to allow the robot to move over the interface        surface;    -   (2) a sensing device which senses at least some of the coded        data and generates indicating data indicative of the identity of        the interface surface and of a position of the robot on the        interface surface;    -   (3) a processor adapted to:        -   (A) select and execute a computer application based on at            least the identity of the interface surface; and,        -   (B) generate movement instructions;            whereby, the behaviour of the robot is at least in part            controlled by the selected computer application.

In a further aspect the present invention provides a robot adapted tooperate in association with an interface surface having disposed thereinor thereon coded data indicative of an identity of the interface surfaceand of a plurality of reference points of the interface surface, therobot comprising:

-   -   (1) at least one motor and at least one drive mechanism to        propel the robot over the interface surface;    -   (2) at least one motor controller to control the motor;    -   (3) a sensing device which senses at least some of the coded        data and generates indicating data indicative of the identity of        the interface surface and of a position of the robot on the        interface surface;    -   (4) a radio transceiver to:        -   (A) transmit the indicating data to a computer system, the            computer system programmed to execute a program based on at            least the identity of the interface surface; and,        -   (B) receive movement instructions from the program;            whereby, the behaviour of the robot is substantially            controlled by the program.

Preferably the program is selected from a plurality of programs on thecomputer system.

In a further aspect the present invention provides a robot adapted tooperate in association with an interface surface having disposed thereinor thereon coded data indicative of an identity of the interface surfaceand of a plurality of reference points of the interface surface, therobot comprising:

-   -   (1) at least one motor and at least one drive mechanism to        propel the robot over the interface surface;    -   (2) at least one motor controller to control the motor;    -   (3) a sensing device which senses at least some of the coded        data and generates indicating data indicative of the identity of        the interface surface and of a position of the robot on the        interface surface;    -   (4) a processor to:        -   (A) select and execute a program based on at least the            identity of the interface surface; and,        -   (B) generate movement instructions;            whereby, the behaviour of the robot is substantially            controlled by the program.

In a further aspect the present invention provides a system forcontrolling the movement of a robot, the system comprising:

-   -   (1) an interface surface having disposed therein or thereon        coded data indicative of an identity of the interface surface        and of a plurality of reference points of the interface surface;    -   (2) a computer system, the computer system programmed to select        and execute a computer application based on at least the        identity of the interface surface, the computer system able to        communicate with the robot;    -   (3) the robot adapted to operate in association with the        interface surface, the robot including:        -   (A) movement means to allow the robot to move over the            interface surface;        -   (B) a sensing device which senses at least some of the coded            data and generates indicating data indicative of the            identity of the interface surface and of a position of the            robot of the interface surface; and,        -   (C) communication means to transmit the indicating data to            the computer system and receive movement instructions from            the selected computer application;            whereby, the behaviour of the robot is at least in part            controlled by the selected computer application.

In another aspect the present invention provides a method of controllingthe movement of a robot, the robot adapted to operate in associationwith an interface surface having disposed therein or thereon coded dataindicative of an identity of the interface surface and of a plurality ofreference points of the interface surface, the robot additionallyprovided with movement means to allow the robot to move over theinterface surface and a sensing device which senses at least some of thecoded data and generates indicating data indicative of the identity ofthe interface surface and of a position of the robot on the interfacesurface, the method including the steps of:

-   -   the robot transmitting the indicating data to a computer system,        the computer system programmed to select and execute a computer        application based on at least the identity of the interface        surface; and,    -   the robot receiving movement instructions from the selected        computer application.

In a further aspect the present invention provides a method ofcontrolling the movement of a robot, the robot adapted to operate inassociation with an interface surface having disposed therein or thereoncoded data indicative of an identity of the interface surface and of aplurality of reference points of the interface surface, the robotadditionally provided with movement means to allow the robot to moveover the interface surface and a sensing device which senses at leastsome of the coded data and generates indicating data indicative of theidentity of the interface surface and of a position of the robot on theinterface surface, the method including the steps of:

-   -   the robot processing the indicating data in a processor, the        processor executing a program based on at least the identity of        the interface surface; and,    -   the processor providing movement instructions from the program.

In a second aspect the present invention provides a robot adapted tooperate in association with an interface surface having disposed thereinor thereon coded data indicative of a plurality of reference points ofthe interface surface, the robot comprising:

-   -   movement means to allow the robot to move over the interface        surface;    -   a sensing device which senses at least some of the coded data        and generates indicating data indicative of a position of the        robot on the interface surface;    -   communication means to transmit the indicating data to a        computer system running a computer application, and to receive        movement instructions from the computer application; and,    -   a marking device adapted to selectively mark the interface        surface in response to marking instructions received from the        computer application.

In a further aspect the present invention provides a system for markingan interface surface by controlling a robot provided with means to markthe interface surface, the system comprising:

-   -   the interface surface, having disposed therein or thereon coded        data indicative of a plurality of reference points of the        interface surface, thereby facilitating the robot to operate in        association with the interface surface;    -   the robot, provided with movement means to allow the robot to        move over the interface surface, and a sensing device which        senses at least some of the coded data and generates indicating        data indicative of a position of the robot on the interface        surface, and a marking device adapted to mark the interface        surface; and,    -   a computer system running a computer application, the computer        system in communication with the robot, the computer application        adapted to receive the indicating data and to transmit movement        instructions and marking instructions to the robot.

In another aspect the present invention provides a method of marking aninterface surface by controlling a mobile robot adapted to mark theinterface surface, the robot additionally adapted to operate inassociation with the interface surface having disposed therein orthereon coded data indicative of a plurality of reference points of theinterface surface, the method including the steps of:

-   -   the robot sensing at least some of the coded data and generating        indicating data indicative of a position of the robot on the        interface surface;    -   the robot transmitting the indicating data to a computer system        running a computer application;    -   the robot receiving movement instructions from the computer        application; and,    -   the robot receiving marking instructions from the computer        application to selectively mark the interface surface.

BRIEF DESCRIPTION OF FIGURES

The present invention should become apparent from the followingdescription, which is given by way of example only, of a preferred butnon-limiting embodiment thereof, described in connection with theaccompanying figures.

FIG. 1 shows a plan view and two elevations of a mobile robot 230according to a preferred form of the present invention;

FIG. 2 shows a plan view and two elevations of the internal mechanicalstructure of the mobile robot 230 of FIG. 1;

FIG. 3 shows a layout of an interactive printed chess board 250according to a preferred form of the present invention;

FIG. 4 shows a block diagram of a mobile robot controller 200 and itsassociated peripherals, according to a preferred form of the presentinvention;

FIG. 5 shows a communication flow between the mobile robot 230 andassociated networked entities; and

FIG. 6 shows an application context class diagram, in UML notation, fora robot control application according to a preferred form of the presentinvention.

MODES FOR CARRYING OUT THE INVENTION

The following modes are described in order to provide a more preciseunderstanding of the subject matter of the present invention.

In the figures, incorporated to illustrate the features of the presentinvention, like reference numerals are used to identify like partsthroughout the figures.

Robot With Code Sensor

A mobile robot is typically an electro-mechanical device incorporating acontroller and a steerable propulsion system. It may incorporate sensorsfor reacting to its environment. For example, it may incorporate one ormore contact sensors so that when it bumps into an obstacle it can takeremedial action such as reversing and proceeding in a differentdirection. A more sophisticated mobile robot may have access to a map ofits environment, and may navigate from one point to another withreference to the map. It may also create a map of its environment duringits explorations, either via its contact sensors, or via a moresophisticated sensory or vision system. Principles of mobile roboticsare described in detail in Dudek, G., and M. Jenkin, ComputationalPrinciples of Mobile Robotics (Cambridge University Press, 2000) andNehmzow, U., Mobile Robotics: A Practical Introduction (Springer Verlag,2000), the contents of both of which are incorporated herein bycross-reference. Practical mobile robot construction is described indetail in Wise, E., Applied Robotics (Prompt Publications, 1999), andMcComb, G., The Robot Builder's Bonanza (Second Edition, McGraw Hill,2001), the contents of both which are incorporated herein bycross-reference.

The robot 230, a preferred form of which is illustrated in FIG. 1 andFIG. 2, is a mobile robot incorporating a differential drive systemsteered by an on-board controller 200 and is hereinafter referred to asa netpage robot. The drive system includes a pair of DC motors 206, witheach motor coupled to a wheel 208 via a gearbox 207. Each motor iscontrolled by a motor controller 205, such as a discrete or integratedH-bridge to provide direction and braking control, and a pulse-widthmodulator to provide speed control. The drive system may alternativelyutilize a pair of stepper motors 206, in which case each motorcontroller 205 generates the appropriate pulse sequences to control thespeed and direction of its corresponding stepper motor. A castor 209provides a third point of support for the robot. The robot incorporatesa platform 211 to which the motors and gearboxes are attached, and whichis itself attached to the robot housing 210 (shown enclosing the robotin FIG. 1 and in outline in FIG. 2). The platform 211 also supports themain PCB 212 which holds the controller 200 and other electroniccomponents.

The netpage robot has a wireless communications link to a host computersystem. It utilizes a radio transceiver 189 (e.g. operating according toa short-range standard such as Bluetooth, or a longer-range standardsuch as GSM, GPRS or other CDMA variant). Alternatively it may utilizean infrared transceiver (e.g. operating according to the IrDA standard).It is powered by a rechargeable battery 139.

When placed on a netpage, the netpage robot 230 determines, via itsin-built netpage sensor, the id 50 of the netpage and its own position(and orientation) 221 relative to the netpage. It then reports thisinformation to the host computer via the communications link. Thenetpage sensor incorporates infrared illumination LED(s) 131, an areaimage sensor 132, and optics 135 including an infrared filter, apertureand focussing optics. The robot detects contact with a surface viacontact switch 194, or alternatively by successfully sensing anddecoding the surface coding pattern.

The robot has a unique id 220 which it communicates to the host andwhich identifies it uniquely among netpage robots. The robot isresponsive to a command 222 from its host to move to a specifiedposition on a specified netpage. When the robot reaches its targetposition it reports its arrival to the host. Because the robot receivescontinuous position information from its netpage sensor, it is able tonavigate with absolute accuracy, without the cumulative errors whichplague odometry systems (due to such things as wheel slip and inaccuratekinematic modelling), and therefore without the need to include moresophisticated position-correction mechanisms.

To allow the robot to move with maximum efficiency, a movement commandis usefully specified in terms of a polyline or piece-wise curve. Thisallows the robot to follow a multi-segment path without slowing to astop between segments. The robot may also be responsive to overlappedcommands, where it is able to receive and buffer one or more newcommands before completing a current command. In the case of a polyline,the robot may optionally be instructed to interpolate a smooth curvedpath through the vertices of the polyline.

Commands to the robot may specify speed, or the robot may be allowed tochoose its own speed according to its capabilities and the complexity ofthe path. If the speed is specified, then the polyline (or piece-wisecurve) may contain a number of speed specifications (e.g. at eachvertex), which are interpolated by the robot.

The size of the netpage robot may vary from small (e.g. millimeter orcentimeter scale) to large (e.g. decimeter or meter scale). The netpagecoded data pattern may be scaled to match the scale and intendedapplications of the robot. However, a single coded data pattern scalemay be used across the full range of robot scales.

Page-Driven Robot Behavior

For the purposes of the following discussion, the game of chess is usedfor illustrative purposes. Note, however, that the principles aregeneral and apply to games other than chess, and to applications beyondgames.

The netpage chess board 250, as shown in FIG. 3, may be pre-made or maybe printed on demand using a netpage printer. The board incorporates aplaying area 254, consisting of an 8×8 checkerboard pattern, as well asadditional areas and controls. When a netpage robot is placed on thenetpage chess board, it initially reports its own position andorientation and the netpage id 50 of the netpage chess board to the hostcomputer. The page server 841 on the host computer identifies the chessapplication associated with the netpage id of the netpage chess board(or associated with a zone of a field embedded in the netpagedescription associated with the netpage id), and forwards all robotinput to the chess application. The host computer also forwards robotcontrol commands from the chess application to the robot. The chessapplication may be local to the host computer, or may be executing on adifferent computer on a local-area or wide-area network (e.g. theInternet).

Since chess involves sixteen pieces, the chess application receivessixteen initial position reports from sixteen participating robots. Ifit receives less than sixteen initial position reports then it cansignal an error to the user, e.g. via an audible or printed output. Ifit receives more than sixteen initial position reports, then it caninstruct the redundant robots to move off the chess board. The chessapplication assigns each robot the role of one of the sixteen chesspieces, and instructs each robot to move to its starting position. Thechess application must plan each robot's path to avoid collisions withother robots. For chess, a simple heuristic consists of the following:

-   -   assign each back-line (i.e. non-pawn) role to the nearest robot    -   assign each front-line (i.e. pawn) role to the nearest remaining        robot    -   move to its starting position any robot not yet in its starting        position which has an un-obscured straight-line view of its        starting position    -   repeat until all pieces are properly positioned or no more        pieces can be moved    -   move any remaining pieces using two-part moves

Clearly more sophisticated algorithms are possible, many of which resultin more rapid positioning.

The user makes a move by picking up a robot and placing it at a newposition. The robot reports its new position to the chess application.If the move is the first move of a game then the “color” of the robotdetermines whether the user has chosen to play as white or black. Eachrobot ideally incorporates some kind of display 186 to indicate whichrole it has been assigned (e.g. white queen or black king). The displaymay be in the form of a small alphanumeric LCD showing an alphanumericrepresentation of the role, or a monochrome or color bit-mapped LCD orOLED showing an icon of the role. The robot is responsive to a commandfrom the host (i.e. from the chess application via the host) to programthe content of the robot's display. The robot may have multipledisplays. For example it may have one display on its top and another onits face. Its entire visible surface may be completely covered by one ormore displays, allowing a semi-three-dimensional icon to be displayed onthe robot.

The chess application may chose to position the moved robot morecentrally in its square, to allow more uniform navigation of otherrobots. It may also normalize its orientation, so that it is alwaysfacing the opposition. If the move is illegal, or if the user mistakenlymoves an opposing piece, then the chess application can issue a spokenwarning (e.g. via the robot's speaker 204, display 186, or status LED(s)116) and reverse the move. If a piece is accidentally placed off theboard, then the chess application can once again issue a warning.

The chess application then computes its responding move, and instructsone or more of its robots to move accordingly. Chess-playing algorithmsand software are described in detail in Pawlak, R. J., Chess SoftwareSourcebook (Treehaus Books, 1999) and Heinz, E. A., Scalable Search inComputer Chess: Algorithmic Enhancements and Experiments at High SearchDepths (Morgan Kaufmann, 1999), the contents of both of which areincorporated herein by cross-reference.

The user can also play against a remote human opponent, in which caseeach local move by the user is tracked by the chess application, and ismirrored, under chess application control, by the corresponding remotepiece.

In the case of pieces which are allowed to move despite being hemmed in,such as knights, or pieces which are allowed to move diagonally, such asbishops and queens, the chess application may temporarily move offendingrobots out of the way, irrespective of their color. It may alsotemporarily move robots off the playing area 254, e.g. during castling.The chess application maintains a map of the chess board for navigationpurposes. This may be the same map it uses for game-playing purposes, orseparate from it. When the user captures an opposing piece s/he simplyplaces the piece off the playing area. When the chess applicationcaptures an opposing piece (i.e. one of the user's pieces), it instructsthe piece to move off the playing area. The chess board includes an area251 for out-of-play pieces of each color. The chess application canautomatically move a piece captured by the user to the out-of-play area,irrespective of where on the board the user places the piece aftercapture. If the chess board is upset and some pieces lose their places,then the chess application can automatically return each piece to itsproper place. Alternatively the chess board can include a <resume> zone(not shown) which, when activated by the user with a piece (or othernetpage sensing device), can instruct the chess application to returneach piece to its proper place. To allow the user to resign, the chessboard includes a <resign> zone 253 which, when activated by the userwith a piece (or other netpage sensing device), or specifically with theresigning king, signals the chess application that the user isresigning. The chess application signals that it is resigning by movingits king off the playing area, or specifically to its own <resign> zone.The board also includes a <new game> zone 252 which, when activated bythe user with a piece (or other netpage sensing device), signals thechess application to abandon the current game and set up a new game.There may be multiple <new game> zones corresponding to different levelsof game-play by the chess application, e.g. ranging from beginnerthrough intermediate to advanced.

When a pawn reaches the opposing back line and is transformed into aqueen, the chess application re-assigns the role of the robot, andupdates the robot's display(s) accordingly.

The netpage robot bay holds a number of netpage robots. The surface ofthe bay is netpage position-coded to allow robots to navigate into andwithin the bay under host control. The bay incorporates a netpage sensorwhich allows it to detect when and where it is placed on a netpage, anda communications link to the host computer which allows it to report theid of the netpage it is placed on and its own position and orientationrelative to the netpage. When the bay is placed on a netpage on whichone or more netpage robots are located, the host computer canautomatically instruct the robots to enter the bay. To this end the bayhas a gentle ramp at its entry which a netpage robot is able tonavigate. The chess board (or bay, or both) may also include a <gather>zone which, when activated by the user with a piece (or other netpagesensing device), signals the host to gather the robots in the bay.

The netpage robot bay incorporates a battery charging circuit forre-charging the batteries of the robots in the bay. The bay has anexternal power connection, and may incorporate a larger storage batteryfor offline operation. A robot may be automatically re-charged in thebay, or may re-charge in response to a re-charge instruction from thehost.

The netpage robot may additionally incorporate a directional contactsensor, allowing it to detect and report to the host the presence of anyunexpected obstacles. Such obstacles are “netpage-invisible” if they donot themselves incorporate netpage sensors or do not report theirpresence (and position) to the host. Any netpage robot is by definitionnetpage-visible and can therefore not become an unexpected obstruction.

To support games such as checkers which allow pieces to be stacked, thenetpage robot is designed to be stacked. To this end its top surface issurrounded by a rim which is designed to mate with the undercarriage ofanother robot (this detail is not included in FIG. 1 and FIG. 2).

To support games such as checkers and go which only involve two kinds ofpieces, the netpage robot may incorporate a simpler display or indicatorsystem (such as an LED or a two-color LED), which can indicate which ofthe two roles the robot has been assigned.

There are many alternatives to the architecture described above, i.e.where a host computer mediates communication between robots and a(possibly remote) chess application. For example, one of the robots maybe automatically designated as the master robot for the duration of thegame (or longer), and may download the chess application for executionon its embedded processor. In this case the master robot controls theslave robots. As another example, the robots may act as peers, and mayeach download the chess application (or parts of it) and execute itcollaboratively, communicating directly among themselves.

The key feature of the netpage robot system architecture is that thebehavior of a collection of one or more robots derives from theparticular netpage on which they are placed. When placed on a chessboard the robots behave as chess pieces; when placed on a checkers boardthey behave as checkers pieces; etc. This is enabled by the netpage id(either directly or indirectly) identifying the application, theapplication controlling the robots (either remotely or embedded), andthe robots navigating absolutely using the netpage position coding.

Marking Robot

The netpage robot may incorporate a marking device for marking a surfacesuch as a paper surface. The marking device is under robot control, sothat the robot can turn marking on and off in a position-dependent way.This may be effected using a digitally-controlled marking device 195such as a Memjet™ printhead, or with a more traditional pen which can bemoved into contact with the surface under robot control. The robot cansupport multiple colors via a multi-color printhead, or via multipleprintheads, or via multiple colored pens. Suitable linear printheads aredescribed in co-pending PCT applications WO 01/89839 entitled “Ink JetPrinthead Having a Moving Nozzle with an Externally Arranged Actuator”,filed 19 Oct. 2001, WO 01/89844 entitled “Ink Jet Printhead NozzleArray”, filed 24 May 2000, and related applications incorporated bycross-reference therein and herein. Suitable circular printheads aredescribed in co-pending PCT application WO 02/34531 entitled “Printheadfor Pen”, filed 19 Oct. 2001.

Multiple robots can also be used, each carrying a differently-coloredpen. These can be controlled sequentially or simultaneously to drawdifferent parts of a multi-colored drawing.

When the user inserts a new pen cartridge into a robot, the robot mayautomatically determine the color of the pen by decoding a barcode onthe surface of the cartridge, or by reading a value from a solid-statememory embedded in the cartridge. Alternatively, the user may activate acolor-coded netpage hyperlink using the robot (or other netpage sensingdevice), thus notifying the computer system and/or correspondingapplication of the robot's new pen color. A palette of such color-codedhyperlinks may be provided on a separate general-purpose netpage or maybe incorporated into the design of an application-specific netpage.

The marking robot may incorporate one or more printheads with asignificant width, e.g. of the order of the diameter of the robot itself(as shown in FIG. 2). This allows the robot to draw lines with subtlestructure, such as varying width and varying color texture. It alsoallows the robot to operate in raster mode, as opposed to (or inaddition to) the vector mode described so far. In raster mode the robotcan transfer a two-dimensional image to a surface in a single sweep, orbuild up a larger image in multiple sweeps. The source image may be abit-mapped image such as a photograph, or it may be a set of vectorstrokes, e.g. corresponding to handwritten text. Reproducing handwrittentext in raster mode may be significantly faster than reproducing it invector mode, particularly if the height of the text is less than thewidth of the printhead(s). In raster mode the computer system (or therobot) keeps track of exactly which parts of the source image have beentransferred to the surface, allowing the robot to transfer the image tothe surface without gaps or overlaps between successive sweeps.

The marking robot, in addition to being responsive a command from itshost to move to a specified position on a specified netpage, is alsoresponsive to a command from its host to draw a line of a specifiedcolor while moving to the target position. As before, the host mediatescommunication between the robot and an application which is specific tothe netpage (or netpage region) on which the marking robot is placed.The host may hide the drawing functions of the robot behind aline-drawing interface which it presents to the application. Theapplication is then unaware of how the drawing functions areimplemented. The robot may directly support the drawing of curved lines,i.e. during movement along curved paths, or the application (or host)may approximate curved lines by instructing the robot to draw asuccession of shorter straight-line segments.

The line-drawing interface, or the robot itself, may be responsive toPostScript drawing commands (refer to Adobe Systems Incorporated,PostScript Language Reference (3rd Edition, Addision-Wesley 1999), thecontents of which are incorporated herein by cross-reference).

When the marking robot marks a netpage, the markings persist as part ofthe netpage in the usual way. In this way they become persistent,reproducible, and searchable.

Applications of the marking robot include (but are not limited to): pureplotting of drawings, particularly where it is impractical for the userto utilize a large-format printer, but it is practical for the user toobtain a large-format pre-netpage-coded “blank”; collaborative markup,where each participant's contribution, typically hand-drawn orhandwritten with a netpage pen, is robotically mirrored on each of theother participants' “work-in-progress” netpages; and system feedback inthe absence of a page-oriented printer.

Remote Conferencing And Collaborative Markup

As described in the context of the chess application, when the userplaces one or more robots on a collaborative markup netpage thecorresponding collaborative markup application can automatically assigneach of the robots to represent one of the remote collaborativeparticipants. If names, icons and/or photos representing theparticipants are available, then this information can be displayed onthe robots. In the limit case, each robot's display can act as a videoconferencing display for the remote participant it represents, i.e.continuously displaying video of the remote participant capturedremotely via a video camera and transmitted over the network. The robotmay also usefully incorporate a speaker 204 for relaying the remoteparticipant's voice, and a microphone 201 for capturing voice input. Themicrophone may also be provided separately, e.g. attached to the hostcomputer (or to the relay—see below).

A remote conferencing netpage application is described in co-pending PCTapplication WO 01/02940 entitled “Method and System for Conferencing”,filed 30 Jun. 2000. A collaborative document markup netpage applicationis described in co-pending PCT application WO 01/02939 entitled “Methodand System for Collaborative Document Markup”, filed 30 Jun. 2000.

Mirroring Robot Movement

In the same way that a netpage robot can mirror the movement of a remoteuser's netpage pen, a robot can mirror the movement of a remote robot.The remote robot may be under direct or indirect user control, i.e. itmay be being moved by the user's own hand, or it may be being movedunder interactive computer control. For example, in an interactive tankgame being played by multiple remote participants, each user may controltheir local tank via a graphical user interface on a PC. The tank mayinclude a video camera for capturing a tank's-eye view of the gameenvironment, and the video can be shown on the user's PC displaytogether with various tank navigation controls. Alternatively the videocan be shown on a conventional television set receiving the videowirelessly from the tank, independently of the computer system. In thiscase the user can also control the tank via a printed netpage tankcontrol user interface and a netpage stylus, i.e. without any directinteraction between the user and a PC.

Each user may have an additional local tank robot representing eachremote user's tank. Each such tank simply mirrors the correspondingremote tank's movements, under the control of the network-resident tankgame application.

The robots may have a game-specific shape (such as the shape of a tank),or may be generic. They may also be general-purpose robots which can beinserted into low-cost game-specific bodies (e.g. tanks) for thepurposes of specific games.

Motion Recording And Playback

As mentioned above, a user may directly move a robot, i.e. by hand, andhave the movement mirrored by a remote robot. The user may also instructa robot to record a hand-specified movement, and then have the robotreplay the recorded movement. Recording (and playback) may be initiatedby depressing a button on the robot, or by providing a control inputthrough some other interface, such as by designating a <record> zone (or<playback> zone) on a netpage with a netpage sensing device (includingthe robot itself), or by specifying a command by other means, such askeyboard or mouse input to a computer system.

Motion recording and playback may be provided as a generic function ofthe host computer system, or may be provided by specific applications.In either case recording and playback rely on already-describedmechanisms. Hand-specified motion may optionally be subject to smoothingby the computer system before playback, and a robot may also beinstructed to follow a path at a different speed to the varying speedassociated with the original hand-specified motion.

Robotic Viewer

A netpage viewer is a device which, when placed on a netpage, provides aregistered interactive view of elements of the page underlying theviewer. The viewer incorporates a netpage sensor for determining its ownposition and orientation relative to the netpage, and to allow it toobtain interactive content specific to the page from the netpage system.A netpage viewer is described in co-pending PCT application WO 01/41046entitled “Viewer with Code Sensor”, filed 27 Nov. 2000. A netpage viewerincorporating a printer is described in co-pending PCT application WO01/41045 entitled “Viewer with Code Sensor and Printer”, filed 27 Nov.2000.

A netpage viewer is usefully augmented with robotic capabilities, toallow the viewer to mirror movement of a remote viewer, or to allow theviewer to display remote netpage pen input on its display, wherever theinput may appear on the page. The robotic netpage viewer will move tothe area of the page where input is occurring, to allow the input to bedisplayed in registered fashion. This is particularly useful in acollaborative markup context as described above. The user may at anytime move the viwer to a different part of the page to view earliermarkup. If the viewer incorporates a printer, then remote (and local)markup can be simultaneously (and persistently) transferred to theunderlying physical page.

Robot Controller

FIG. 4 shows a block diagram of the robot controller 200 and itsassociated peripherals. The integrated controller chip incorporates aprocessor 145 connected to a number of other components via a shared bus146, including a working memory 148 (such as an SRAM). An off-chipnon-volatile memory 147 (such as a flash memory) holds the fixed robotprogram and other persistent data. The controller includes a parallelinterface for controlling the motor controllers 205, status LED(s) 116,and illumination LED(s) 131, and for sensing the optional contact switch194. It includes a serial interface for communicating with a hostcomputer (and/or other robots) via the radio transceiver 189, and forcontrolling the optional printhead 195. It includes an image sensorinterface 152 for the netpage sensor's image sensor 132, an optionaldisplay interface 185 for the display 186, an optional ADC 202 for themicrophone 201, and an optional DAC 203 for the speaker 204.

Robot Communication Flow

FIG. 5 shows a possible communication flow between a netpage robot 230and associated networked entities in the context of the netpage system.

When the robot is placed on a netpage, its goal is to notify itspresence to a corresponding robot control application 804, and beginreceiving commands from the application. The robot typically hasshort-range wireless connectivity with a relay device 44, which itselftypically has longer-range network connectivity to netpage systemservers via wired LAN, WAN or a longer-range wireless link. The relaydevice may be a mobile phone, as described in co-pending PCT applicationWO 00/72286 entitled “Relay Device”, filed 24 May 2000; an interactivenetpage printer, as described in co-pending PCT application WO 02/42894entitled “Interactive Printer”, filed 26 Nov. 2001; etc.

The robot transmits its initial position report to the relay, includingits robot id 220, the netpage id 50 of the page on which it findsitself, and its position (and orientation) 221 on the page. The relay isable to determine, with reference to one or more id servers 12, theserver id 53 of the page server 841 which “owns” the page description ofthe page. Alternatively, the robot may itself be able to determine theserver id 53 from the id server(s) 12, transparently via the relay 44.The robot may also implicitly incorporate the relay function if it hasits own longer-range network connectivity.

The relay (or robot) then forwards the position report to the pageserver 841. The page server in turn determines which application 804 toforward the position report to. In the general netpage scheme, thisconsists of hit-testing the robot position against the variousinteractive elements embedded in the description of the page. For thepurposes of robot control, all robot movement is usefully forwardeddirectly to the application without further interpretation by the pageserver. It is also possible to dispense with the page server entirely,and forward a robot position report directly from the robot (or relay)to an application as identified by the id server(s), or even asidentified directly by the page id 50 embedded in the original page.Note that while the ids used in the netpage system are typicallydescribed as “pure” ids which do not support routing per se, it is alsopossible, for example, for these ids to be partial or complete InternetProtocol (IP) addresses, either in 32-bit (IPv4) or 128-bit (IPv6) form,thus directly supporting routing.

The application 804 eventually receives the robot position report by oneof these mechanisms, and then interprets the position report withreference to the layout and semantics of the page identified by the pageid 50. Since the application may be specific to a particular page layout(e.g. to the chess board), it may be able to assume the layout andsimply use the page id to index a particular instance (e.g. a particularin-progress chess game). The application is then able to send commands222 directly (or indirectly) to the robot, and receive further positioninput from the robot.

As mentioned earlier, there are many possible variations on thisarchitecture, including downloading application logic into one or morerobots, and distributed peer-to-peer robot control.

Application Context Class Diagram

FIG. 6 shows a generic application context class diagram, in UMLnotation, for a robot control application. The overall context 260 (e.g.corresponding to the chess application) includes a number of individualinstances 261 (e.g. corresponding to individual chess games), each ofwhich includes a number of robot roles 262 (e.g. corresponding toindividual chess pieces' roles). The context also includes a number ofknown robots 263, each of which may optionally have been assigned a role262. Each instance 261 is identified by an instance id 265. The instanceid may correspond to the netpage id of the corresponding page, or maycorrespond to a transaction id, embedded in the page description of thecorresponding page, which gets forwarded to the application during robotinteractions. Each role 262 is identified by a role id 266, and has astatus 267 (e.g. indicating whether a chess piece is still in play orhas been captured). Each robot 263 is identified by a robot id 220, i.e.the robot id of the corresponding physical robot 230, and a currentposition 269 derived from the most recent position 221 reported by therobot.

The present invention has been described with reference to a preferredembodiment and number of specific alternative embodiments. However, itwill be appreciated by those skilled in the relevant fields that anumber of other embodiments, differing from those specificallydescribed, will also fall within the spirit and scope of the presentinvention. Accordingly, it will be understood that the invention is notintended to be limited to the specific embodiments described in thepresent specification, including documents incorporated bycross-reference as appropriate.

The invention may also be said to broadly consist in the parts, elementsand features referred to or indicated herein, individually orcollectively, in any or all combinations of two or more of the parts,elements or features, and wherein specific integers are mentioned hereinwhich have known equivalents in the art to which the invention relates,such known equivalents are deemed to be incorporated herein as ifindividually set forth.

Although the preferred embodiment has been described in detail, itshould be understood that various changes, substitutions, andalterations can be made by one of ordinary skill in the art withoutdeparting from the scope of the present invention.

1. A mobile robot for sensing and decoding a printed coding pattern on asurface, the mobile robot comprising a housing; a platform in thehousing; an on-board controller mounted on the platform and configuredfor wireless communication with a host computer system; a differentialdrive system mounted on the platform and connected to the controller sothat the controller can steer the platform relative to the surface; andan image sensor mounted on the platform, arranged to sense and decodethe printed coding pattern, said image sensor system being connected tothe controller to communicate data representing the decoded codingpattern to the host computer via the controller, wherein said printedcoding pattern encodes, at each of at least a given density of points onthe surface, the two-dimensional coordinates of the point and identifiesthe identity of the surface, wherein the on-board controller isconfigured to be responsive to a command from the host computer to moveto a specified coordinate position on the surface, the controllerfurther being configured to report arrival at said specified coordinateposition to the host computer.
 2. A mobile robot as claimed in claim 1,in which the differential drive system includes a pair of wheels, anH-bridge connected to respective wheels to provide direction and brakingcontrol and a pulse-width modulator to provide speed control.
 3. Amobile robot as claimed in claim 1, in which the differential drivesystem includes a pair of wheels, stepper motors connected to respectivewheels and a motor controller configured to generate appropriate pulsesequences to control speed and direction of the stepper motors.
 4. Amobile robot as claimed in claim 1, in which the on board controller andimage sensor are configured so that, when placed on the surface, thecontroller can determine a position and orientation of the platformrelative to the surface and can communicate associated data to the hostcomputer.
 5. A mobile robot as claimed in claim 1, in which the imagesensor includes infrared illumination LEDs, an area image sensor andoptics all suitable for sensing infrared ink.
 6. A mobile robot asclaimed in claim 1, in which the controller is configured to store aunique ID and to communicate that ID to the host computer.